Geometric Algebra Model of Distributed Representations

Formalism based on GA is an alternative to distributed representation models developed so far --- Smolensky's tensor product, Holographic Reduced Representations (HRR) and Binary Spatter Code (BSC). Convolutions are replaced by geometric products, interpretable in terms of geometry which seems to be the most natural language for visualization of higher concepts. This paper recalls the main ideas behind the GA model and investigates recognition test results using both inner product and a clipped version of matrix representation. The influence of accidental blade equality on recognition is also studied. Finally, the efficiency of the GA model is compared to that of previously developed models.Comment: 30 pages, 19 figure

PIPS: A parallel planning model of sentence production

Subject–verb agreement errors are common in sentence production. Many studies have used experimental paradigms targeting the production of subject–verb agreement from a sentence preamble (The key to the cabinets) and eliciting verb errors (… *were shiny). Through reanalysis of previous data (50 experiments; 102,369 observations), we show that this paradigm also results in many errors in preamble repetition, particularly of local noun number (The key to the *cabinet). We explore the mechanisms of both errors in parallelism in producing syntax (PIPS), a model in the Gradient Symbolic Computation framework. PIPS models sentence production using a continuous-state stochastic dynamical system that optimizes grammatical constraints (shaped by previous experience) over vector representations of symbolic structures. At intermediate stages in the computation, grammatical constraints allow multiple competing parses to be partially activated, resulting in stable but transient conjunctive blend states. In the context of the preamble completion task, memory constraints reduce the strength of the target structure, allowing for co-activation of non-target parses where the local noun controls the verb (notional agreement and locally agreeing relative clauses) and non-target parses that include structural constituents with contrasting number specifications (e.g., plural instead of singular local noun). Simulations of the preamble completion task reveal that these partially activated non-target parses, as well the need to balance accurate encoding of lexical and syntactic aspects of the prompt, result in errors. In other words: Because sentence processing is embedded in a processor with finite memory and prior experience with production, interference from non-target production plans causes errors

Low-cost representation for restricted Boltzmann machines

This paper presents a method for extracting a low-cost representation from restricted Boltzmann machines. The new representation can be considered as a compression of the network, requiring much less storage capacity while reasonably preserving the network's performance at feature learning. We show that the compression can be done by converting the weight matrix of real numbers into a matrix of three values {-1, 0, 1} associated with a score vector of real numbers. This set of values is similar enough to Boolean values which help us further translate the representation into logical rules. In the experiments reported in this paper, we evaluate the performance of our compression method on image datasets, obtaining promising results. Experiments on the MNIST handwritten digit classification dataset, for example, have shown that a 95% saving in memory can be achieved with no significant drop in accuracy

Ground-water Quality in the Bethpage-Hicksville-Levittown Area, Long Island, New York, with Emphasis on Volatile Organic Compounds

A plume of contaminated ground water has been delineated within an 11.4-square-mile area in east-central Nassau County, where residential neighborhoods surround an area zoned for industrial use. The industrial zone contains several firms that, in the past, have discharged effluent containing volatile organic compounds into the upper glacial aquifer through onsite recharge basins. The upper glacial aquifer is in direct hydraulic connection with the underlying Magothy aquifer; the first continuous formation that impedes downward movement of ground water is the Raritan confining unit, which is more than 500 feet below sea level. The chemicals in ground water·and their distribution were identified through analysis of water samples collected from 56 monitoring wells and 11 industrial wells in the spring and fall of 1986 and 1987. Trichloroethylene and tetrachloroethylene were found near the water table at concentrations greater than 1,000 micrograms per liter and were detected more than 5,000 feet downgradient of the industrial zone. The distribution of several other volatile organic compounds indicates more than one contaminant source in the industrial area. The plume of contaminated ground water in 1987 was 12,000 feet long, 5,700 feet wide, and more than 500 feet thick. In 1987, water was pumped from 14 industrial wells, completed in the Magothy aquifer, at an average rate of 8.05 million gallons per day, mainly for cooling purposes. The water was returned chemically unaltered to recharge basins from which it could percolate to the water table. Water-table mounding beneath basins and drawdowns near the pumped wells greatly increase the vertical component of ground-water flow beneath the industrial zone, which has increased the rate of advective movement of contaminated ground water downward toward the screened zones of pumped wells, which are 370 to 560 feet deep in the Magothy aquifer. The concentration of tetrachloroethylene decreases much more rapidly than that of trichloroethylene downgradient of the industrial zone, which indicates that sorption and (or) biodegradation may be occurring. The major effect of industrial activity on inorganic constituents is to decrease the concentration of constituents near the water table in the vicinity of the recharge basins by the addition of water from the Magothy aquifer, where the concentration of inorganic constituents is lower

Geometric representations for minimalist grammars

We reformulate minimalist grammars as partial functions on term algebras for strings and trees. Using filler/role bindings and tensor product representations, we construct homomorphisms for these data structures into geometric vector spaces. We prove that the structure-building functions as well as simple processors for minimalist languages can be realized by piecewise linear operators in representation space. We also propose harmony, i.e. the distance of an intermediate processing step from the final well-formed state in representation space, as a measure of processing complexity. Finally, we illustrate our findings by means of two particular arithmetic and fractal representations.Comment: 43 pages, 4 figure

Universal neural field computation

Turing machines and G\"odel numbers are important pillars of the theory of computation. Thus, any computational architecture needs to show how it could relate to Turing machines and how stable implementations of Turing computation are possible. In this chapter, we implement universal Turing computation in a neural field environment. To this end, we employ the canonical symbologram representation of a Turing machine obtained from a G\"odel encoding of its symbolic repertoire and generalized shifts. The resulting nonlinear dynamical automaton (NDA) is a piecewise affine-linear map acting on the unit square that is partitioned into rectangular domains. Instead of looking at point dynamics in phase space, we then consider functional dynamics of probability distributions functions (p.d.f.s) over phase space. This is generally described by a Frobenius-Perron integral transformation that can be regarded as a neural field equation over the unit square as feature space of a dynamic field theory (DFT). Solving the Frobenius-Perron equation yields that uniform p.d.f.s with rectangular support are mapped onto uniform p.d.f.s with rectangular support, again. We call the resulting representation \emph{dynamic field automaton}.Comment: 21 pages; 6 figures. arXiv admin note: text overlap with arXiv:1204.546

Fewer epistemological challenges for connectionism

Seventeen years ago, John McCarthy wrote the note Epistemological challenges for connectionism as a response to Paul Smolensky’s paper 'On the proper treatment of connectionism'. I will discuss the extent to which the four key challenges put forward by McCarthy have been solved, and what are the new challenges ahead. I argue that there are fewer epistemological challenges for connectionism, but progress has been slow. Nevertheless, there is now strong indication that neural-symbolic integration can provide effective systems of expressive reasoning and robust learning due to the recent developments in the field

Context Dependence, MOPs,WHIMs and procedures Recanati and Kaplan on Cognitive Aspects in Semantics

After presenting Kripke’s criticism to Frege’s ideas on context dependence of thoughts, I present two recent attempts of considering cognitive aspects of context dependent expressions inside a truth conditional pragmatics or semantics: Recanati’s non-descriptive modes of presentation (MOPs) and Kaplan’s ways of having in mind (WHIMs). After analysing the two attempts and verifying which answers they should give to the problem discussed by Kripke, I suggest a possible interpretation of these attempts: to insert a procedural or algorithmic level in semantic representations of indexicals. That a function may be computed by different procedures might suggest new possibilities of integrating contextual cognitive aspects in model theoretic semanti